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In recent years, community structure has emerged as a key component of complex network anal- 
ysis. As more data has been collected, researchers have begun investigating changing community 
structure across multiple networks. Several methods exist to analyze changing communities, but 
most of these are limited to evolution of a single network over time. In addition, most of the ex- 
isting methods are more concerned with change at the community level than at the level of the 
individual node. In this paper, we introduce scaled inclusivity, which is a method to quantify the 
change in community structure across networks. Scaled inclusivity evaluates the consistency of the 
classification of every node in a network independently. In addition, the method can be applied 
cross-sect ionally as well as longitudinally. In this paper, we calculate the scaled inclusivity for a set 
of simulated networks of United States cities and a set of real networks consisting of teams that play 
in the top division of American college football. We found that scaled inclusivity yields reasonable 
results for the consistency of individual nodes in both sets of networks. We propose that scaled 
inclusivity may provide a useful way to quantify the change in a network's community structure. 

PACS numbers: 89.75.Fb, 89.75.Hc, 89.75.Kd 



I. INTRODUCTION 

In recent years, the study of real- world complex net- 
works has increased dramatically; social networks [l], the 
world wide web [2 , and faculty collaboration networks [3 
are among the most commonly studied. 

An offshoot of this expansion has been the study of 
community structure in complex networks. Introduced in 
2002 by Girvan and Newman [4 , the community struc- 
ture of a network helps make sense of the interactions 
between the nodes. In that paper, the authors analyze a 
network consisting of the members of a karate club that 
split into two factions, a faculty collaboration network 
across Europe, and several others. Mathematical com- 
munity analysis was able to identify the two factions of 
the karate club and split the collaboration network into 
several geographically coherent groups. 

In the past few years, more complex datasets have been 
studied, and the change in community structures of net- 
works over time has become an area of focus. Fig. [l] 
shows an example of changing community structure in 
two realizations of a simple network. Such shifts are easy 
to imagine on a larger scale in a complex network with 
many realizations. 

The consistency (or lack thereof) of communities in a 
given network over time can be a good indicator of large- 
scale change in a network. Hopcroft, et al. [5] studied 
the changes in a network consisting of journal articles 
from the NEC CiteSeer database and their references. 
The communities correlated well with certain fields of 
study, and the emergence of new communities over time 
was shown to mirror the emergence of new fields in the 
literature. 

While change in the community structure of a net- 



work has recently become an object of study, many of 
the methods proposed explicitly or implicitly make as- 
sumptions based on the idea of change over time [6^. 
These methods do not lend themselves well to the anal- 
ysis of multiple realizations of a single network. The 
assumption that networks progress from one to the next 
in a linear fashion is not appropriate when the networks 
are not linearly related. 




(a) Network 1 (b) Network 2 



FIG. 1. (Color online) Consider the two networks in (a) and 
(b). Both are realizations of a network with the same un- 
derlying nodes. Several edges change between the two, and 
two nodes switch communities (identified by node color). In a 
large network and across many realizations, it is easy to imag- 
ine such minor shifts occurring many times. The problem at 
hand is how to quantify the consistency of the communities 
identified across several such realizations. 

In addition, the existing methods spend relatively lit- 
tle time on how best to identify the same community 
across several networks; in some cases, it is assumed that 
communities are sufficiently consistent over time to make 
this a moot point. Across networks that are not linearly 
related, this assumption does not hold. The variation 
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in the community structure of multiple networks can be 
significant, so a more rigorous way to quantify the con- 
sistency of community structure is necessary. 

In this paper, we propose a method for comparing the 
consistency of community structure across different real- 
izations of a network, be they the same network over time 
or simultaneous realizations of a single network. In par- 
ticular, we propose a method to describe how consistently 
each node is part of the same community across different 
partitions, which we call scaled inclusivity. This method 
enables us to identify which nodes tend to remain in the 
same community in different network partitions, forming 
a "core" of that community. Likewise, the method also 
allows identification of transient nodes that become part 
of different communities across partitions. 

The remainder of this manuscript is organized as fol- 
lows: Section II describes the analysis algorithm. Section 
III describes the simulated and real data networks used. 
Section IV describes the results of the algorithm applied 
to both simulated and real data, and Section V discusses 
the implications of this work. 



II. METHODS 

Community structure analysis is a difficult problem. 
There are many community detection methodologies, and 
finding better algorithms is an area of ongoing research. 
We use the QCut algorithm, but any community detec- 
tion algorithm could be used. We only evaluate a method 
that places each node into exactly one community, and 
the method we propose here requires that each node be 
in at least one community. The most common metric 
used to evaluate community structure, modularity, was 
introduced by Newman and Girvan flO]: 



M 



(-)' 



(1) 



where m is the number of modules in the network, en 
is the total number of intra-modular edges in module i 
(i = 1, 2, • • • , m) , is the total degree for the vertices in 
i, and M is twice the total number of edges (the sum of 
the degree for all vertices in the network) . Maximizing Q 
has been proven to be an NP-hard problem [TP, mean- 
ing that the only way to guarantee an optimal solution 
is to try all possible solutions. Because this is impracti- 
cal for all but the smallest networks, various algorithms 
have been created to balance optimization of Q with run 
time. In our analysis, we used the QCut algorithm intro- 
duced in [12 . It should be emphasized that we are not 
attempting to evaluate or demonstrate the effectiveness 
of the QCut algorithm, but rather use QCut as a method 
to identify community structure. 

The method described here is used to assess consis- 
tency of partitions on a set of networks assumed to have 
a similar underlying structure or on a series of networks 
with alterations in the structure over time. While the for- 
mer is a collection of multiple realizations of the networks 



without any ordering (e.g., metabolic networks from dif- 
ferent samples or brain connectivity networks from mul- 
tiple subjects), the latter has a particular ordering of 
the networks in which one network is a rewired version 
of the prior one. In both cases, it is assumed that the 
nodes are constant and that there exists a true partition 
of the nodes which may or may not be explicitly known. 



A. Identify best partition 

The first step is to identify the best partition for each 
of n realizations of the network with v nodes. The goal 
of this step is to find a reasonable partition of the net- 
work since each run of the QCut algorithm (like many 
community structure algorithms currently available) may 
produce slightly different partitions. This step starts by 
generating multiple partitions by the QCut algorithm — 
g times on a network Gi (i = 1, 2, • • • , n). Because the 
goal of any modularity-based algorithm is to maximize 
Q (Eq. [l]), the run that produced the partition with the 
highest Q value for the network is chosen as the best par- 
tition for this network, denoted by Q[. It should be noted 
that this step is unnecessary for deterministic community 
structure algorithms. 

For nondeterministic community structure algorithms 
that are not based on optimization of a single parame- 
ter, an alternative method is to choose the partition that 
is most similar to the others. We propose the Jaccard 
similarity index to determine similarity. 

Let Ql be the j-th partition (j = 1, 2, • • • ,^) of net- 
work i. For any pair of partitions and Qf {c ^ d G 
1, 2, • • • , ^), the similarity of the partitions is assessed by 
calculating the Jaccard similarity index 
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where Sc and Sd are sets of node pairs with the same 
community memberships in partitions and , respec- 
tively. For example, in partition Q^, if nodes q and r are 
in the same community (i.e., rriq = m^), then that node 
pair (g,r) is included in the set Sc- The advantage of 
the Jaccard index is that it does not depend on the arbi- 
trary numbering of communities across partitions. Even 
if community 5 in may correspond to community 11 
in Qf, the Jaccard index can assess the similarity be- 
tween partitions without re-assigning community num- 
bers. Calculating Jed for all pairs of partition results in 
matrix J whose c^d-th element is Jed ii c ^ d and if 
c = d. 



B. Assess consistency of classification 

The next step assesses the consistency of a single par- 
tition when compared with the rest of the group. That 
single partition is chosen from the group and is denoted 



by Qr] it is compared to Q- for i = 1, 2, • • • , n, where 
Qi 7^ Qr' To do so, consistency between communities 
in Q'- and Qr is first assessed at the community level. 
Assume Q[ consists of y communities and Qr consists of 
z communities. Then a community- by-community simi- 
larity matrix XiR is calculated, with its p, q-th element 
(p = 1, 2, • • • , y), = 1, 2, • • • , 2:) calculated as 

\A,\ \R,\ 

where is the set of nodes belonging to the p-th com- 
munity in Q[ (i.e., {a\ma = p}) and Rq is the set of nodes 
belonging to the q-th. community in Qr (i.e.,{6|m5 = q}). 
The resulting values range from to 1, where indi- 
cates zero overlap between the two communities and 1 
represents no change in the member nodes. This met- 
ric is preferable to the relative overlap (intersection over 
union) because it yields a lower value in the case of par- 
tial overlap between a large community and a smaller 
community. In other words, it more harshly penalizes 
poor specificity. The similarity matrix XiR is calculated 
for all realizations i = 1, 2, • • • , n where Q[ ^ Qr^ and 
is used to summarize consistent community membership, 
relative to the reference partition, at the nodal level. A 
vector of length v denoted as Vm, with each element cor- 
responding to each node, is used to record the consistency 
in community memberships. We consider three different 
ways to assess this: binary exclusivity, binary inclusivity, 
and scaled inclusivity. 

In binary exclusive classification, a single community 
in each realization Q[ is identified as the best match for 
each community in the reference partition Qr. This is 
done by finding the maximum in each column of the sim- 
ilarity matrix XiR^ corresponding to community q in the 
reference partition. If the column maximum occurs on 
the p-th row, that indicates community p of partition Q • 
corresponds best to community q of the reference parti- 
tion Qr. For all the nodes corresponding to community 
p in partition Q-, a value of 1 is added to the correspond- 
ing elements of the recording vector Vm- This process is 
repeated for all communities in Qr^ and the occurrence 
of the best match is counted at the nodal level. The 
partitions from all realizations Q^ , j = 1,2, •••,n are 
compared against Qr, producing the final value of the 
recording vector Vm with each nodal value summarizing 
the number of times it was "correctly" classified relative 
to the reference partition Qr. Although this approach 
is intuitive, it fails to account for cases where multiple 
communities roughly split a community in the reference 
partition. For example, in time series data, it is possible 
that two communities will merge to form a larger com- 
munity in the reference partition (see Fig. |2|; it seems 
inaccurate to only count the larger of these two commu- 
nities as correctly classified. 

An alternative approach is binary inclusivity, which is 
similar to binary exclusivity. However, two communi- 
ties in one network that evenly split a community in the 
referent network can both be counted as correct classi- 
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(a) Network 1 (b) Network 2 



FIG. 2. (Color online) Depicted are two communities in (a) 
merging into one community in (b) due to increased inter- 
community edges. In binary exclusivity, the blue (dark gray) 
community would be counted as the better match since the 
number of nodes is greater. These nodes would be given a 
value of 1 in the scaled inclusivity map, and the orange (light 
gray) nodes would be given a value of 0. In binary inclusiv- 
ity, both sets of nodes would be given a value of 1. In scaled 
inclusivity, assuming the second network is the reference, the 
blue (dark gray) nodes would all receive a value of | • | = 0.5, 
and the orange (light gray) nodes would receive a value of 
I • I = 0.4, thereby fairly scaling the scores of all nodes. This 
metric also allows for differentiation between merging commu- 
nities as shown here and one community that does not change 
between two networks, which would have a value of 1 for all 
nodes. 

fications. As in binary exclusivity, the similarity matrix 
XiR is used to calculate the similarity of the communities, 
but all communities with some overlap are included. One 
could restrict this to communities with extensive overlap, 
but some threshold would have to be determined for the 
similarity value of the communities in question. For all 
the nodes belonging to the communities identified as the 
best matches, a value of 1 is added to the correspond- 
ing elements in the recording vector Vm- The process is 
repeated for all the communities in the reference parti- 
tions Qr for partitions from all the realizations Q[, as in 
binary exclusivity described above. One disadvantage of 
this method is that a very large community can include 
a part of a small community, but all common elements 
are considered equally correct. In addition, without a 
threshold, all nodes are assigned the same value at every 
step, which yields no useful information. Appropriately 
defining a threshold is a difficult and subjective task, and 
potentially small differences between some communities 
included and excluded are not reflected by assigning bi- 
nary values. 

Finally, scaled inclusivity takes into account any 
matching with any community in the reference partition. 
While binary exclusivity and inclusivity only add a value 
of 1 to nodes corresponding to the matching communi- 
ties, scaled inclusivity adds the value Xf^ to Ap fl Rq 
(i.e., where Xf^ > 0). Thus, how well a node is classi- 
fied in any given realization is scaled based on how well 
its communities in the two networks match. As in bi- 
nary exclusivity and inclusivity, this process is repeated 
for all the communities in the reference partition Qr for 
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partitions from all the realizations Q[. 

In the remainder of this paper, we consider scaled in- 
clusivity because this method best captures the consis- 
tency and change in community organization. 



C. Generate weighted average maps of consistency 

Bias is inherent in the scaled inclusivity of a network 
that is based on a single referent partition Q^. Every 
other partition is compared to this partition; the classi- 
fication of the nodes in Qi? has significant effects on the 
consistency values. To minimize this bias, the best parti- 
tion for each realization {Q[ for i = 1, 2, • • • , n) is selected 
as the referent partition Qr in turn. Thus, n maps are 
computed. These maps are then averaged together using 
group similarity weights from the Jaccard similarity in- 
dex. This serves to minimize the bias of any one network 
while still valuing the partitions that are most similar to 
the rest of the group. 

Recall that matrix J contains the Jaccard similarity 
index for every pair of partitions Q[ and Q'-^ with Jij=0 
where i = j. The column sum of J, Ji = Jij^ is 

calculated, and the resulting vector is normalized to give 
the weights for the weighed average. Thus, the weight 
assigned to each scaled inclusivity map is proportional 
to the summed similarity of that partition to the rest of 
the group. 



E. Generate more informative maps of individual 
communities 

Scaled inclusivity, as described above, has a major lim- 
itation when considering a single community from a sin- 
gle partition. Such an analysis only includes nodes in the 
community of interest, even if every other partition had 
more nodes in the corresponding community. To solve 
this problem, another map can be made separately for 
each community in Qji. The first step is to add the value 
Xf^ to Ap n Rq^ just as above. Secondly, the value Xf^ 
is subtracted from every node in Ap that is not in Rq 
{Ap n Rq). Thus, if Ap and Rq have any overlap, nodes 
in the intersection will have a certain value added, and 
nodes only in Ap will have a negative value with equal 
magnitude subtracted. These values can be summed for 
p = 1, 2, in Q- where z = 1, 2, • • • , n and Q[ ^ Qr 

as long as Rq is held constant. In this way, the negative 
values will not cancel out positive values - positive val- 
ues occur only for nodes in Rq^ and negative values occur 
only for nodes not in Rq. Much as high positive values 
indicate consistent classification in Rq and a community 
in Q[ with significant overlap with Rq^ very negative val- 
ues indicate consistent classification outside of Rq but in 
a community in Q'- with significant overlap with Rq. In 
other words, they are consistently classified in the same 
community as nodes in Rq across the other partitions. 
This is a useful distinction to make in the case the refer- 
ent partition contains a community of interest that lacks 
certain nodes or segments that are included in all other 
partitions. In this case, the values of those nodes would 
be very negative, indicating group consistency in spite of 
absence from the referent partition's community. 



D. Determine most representative partition by 
node 

A further analysis of scaled inclusivity can provide in- 
formation about which realization is best when evaluat- 
ing a certain portion of the network. To calculate this, 
a scaled inclusivity map is generated with each partition 
in turn as the referent partition, as above. From these, a 
map can be generated to show, for each node in the net- 
work, which realization has the highest scaled inclusivity 
value. This indicates that the node's classification in that 
realization is its most consistent classification - that the 
group community structure in this region is best captured 
by this partition. Groups of nodes that are best charac- 
terized by the same partition are of interest, especially 
if they correspond to a single community, because this 
indicates that this partition does a particularly good job 
of characterizing that community for the entire group of 
networks. However, it should be noted that the actual 
scaled inclusivity values should be taken into account: if 
a given area has low scaled inclusivity values, then it does 
not matter what partition best represents this region be- 
cause it is inconsistent across all realizations. 



III. DATA 
A. Simulated Data 

The first data set to be tested consists of 30 simu- 
lated networks using code from [13 . These networks were 
mapped onto a network of the 256 most populous cities in 
the United States to more easily visualize the community 
structure of the network. 

Lancichinetti, et al. [13] describe a method for creating 
unweighted, undirected networks to test various commu- 
nity structure algorithms. A key component of this al- 
gorithm is that degree and community size distributions 
are power laws, as in many real- world networks. 

Parameters of the algorithm are number of nodes (A/"), 
mixing parameter (/i), average degree {kave)i maximum 
degree {kmax)^ minimum community size {smin)^ maxi- 
mum community size {smax)i and the exponents of the 
power law degree and community size distributions (7 
and /3, respectively). The mixing parameter /i is defined 
such that every node shares a fraction l-fi links with 
other nodes in its community and a fraction ji links with 
nodes outside its community. 
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The algorithm computes degree and community size 
distributions based on the input parameters, and nodes 
are assigned a degree. Nodes are then assigned to com- 
munities, and the network is rewired to preserve degree 
and approximate /i. 

The code to run this algorithm was 
made freely available online by the authors 
( http : / / sant o . fort unato . googlepages . com/benchmark , 
tgz). We used this code to generate 30 networks, each 

with N 256, /i 0.35, kave 10, kmax 50, Smin 

= 15, Smax = 51, 7 = 2, and /3 = 1. The network size 
was chosen to be fairly small for computational ease. 
Minimum community size was set to give a relatively 
consistent number of communities (m). The remaining 
parameters were adjusted slightly from the default values 
such that community analysis would show imperfect 
results similar to the community analysis of real-world 
networks. The sizes of the communities for each of the 
30 networks can be seen in Table [H 



TABLE I. Community sizes for all 30 simulated networks, 
listed in ascending order, shown to give the reader some idea 
about the overall community structure. 




FIG. 3. (Color online) Network 30, with the nodes color-coded 
by simulated community. 

The nodes assigned to a given community in any indi- 
vidual network were random. In order to visualize these 
networks and to give the networks some consistency in 
their community structure, the nodes were assigned to 
the 256 most populous cities in the United States ac- 
cording to 2009 census data [14]. Consider the matrix C, 
whose rows q {i = 1, 2, • • • , 256) contain the longitude 
and latitude of a given city i {i = 1,2,- • • ,256). Recall 
that Aj is the set of nodes in community j {j = 1,2,- • •,m) 
and Sj = \Aj\. Let Cj be the set of coordinate pairs q 
{i G {1,2,- • -,256}) such that Cj also has size Sj. For each 
network, each city was manually assigned to a commu- 
nity based on community size Sj and geographical prox- 
imity. Each node ni G Aj is assigned to an arbitrary 
node Ci G Cj, and the cities inherit the links from the 
associated nodes to keep the network structure intact. 

An example of one of the networks (network 30) is 
shown in Fig. [3] with each node color-coded according to 
community membership. We also generated an alterna- 
tive space-filling visualization. The United States Census 
Bureau releases the latitude and longitude of the borders 
of the country [M]. A map was generated using these 
coordinates, and it was converted into a 1200x600 im- 
age. Let Pxy be a pixel in this grid {x G {1,2,- • •, 1200}, 
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40 







y G {1,2,- • -,600}). For every p^y within U.S. borders, 
the Euclidean distance to each of the 256 most populous 
cities was computed, and the pixel was assigned to the 
nearest city. Fig. |4] shows the simulated communities 
for network 30 (a) and for network 3 (b). The commu- 
nities are shown by color-coding the pixels assigned to 
each node. These networks are shown as examples of 
typical community structure. The two are fairly similar, 
but no community is identical between them. It should 
be noted that due to differences in spacing, some nodes 
have very few assigned pixels, while others have many. In 
other words, the area of a community is not necessarily 
indicative of the number of nodes it contains. 
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(a) Simulated Communities for Network 30 




(b) Simulated Communities for Network 3 



FIG. 4. (Color online) Simulated communities for network 30 

(a) and simulated communities from network 3 for comparison 

(b) . Here, the colors indicate membership in a community. 



B. Real Data 

The second set of test data comes from NCAA Col- 
lege Football from the years 1995 to 2009. In this net- 
work, each team that was active in the Football Bowl 
Subdivision (formerly Division I A, hereafter called FBS) 
constitutes a node, and every game played between two 
FBS teams is a link. In each season, teams play eleven or 
twelve regular season games and one or two postseason 
games. Teams are organized into conferences, which usu- 
ally consist of between eight and twelve teams. Teams 
generally play other teams in their own conference more 
often than teams in other conferences. The main ex- 
ceptions are independent schools, which are not mem- 
bers of any conference. In addition, schools may play 
one game each season against teams that are not in the 
FBS. These games were not counted in the network. In 
rare cases, teams may have played two games against the 
same opponent in one season. The second game was not 
counted; there were no weighted links in these networks. 
Some teams joined the FBS during the time period in 
question. Any games played by these teams against FBS 
opponents prior to joining the FBS were not counted in 
the network. For consistency in the analysis, these nodes 
were included in the networks for these seasons, but each 
team belonged to an isolated community until the year 
it joined the FBS. 

This data set lends itself well to network analysis both 



because of its size (N=120) and because the communities 
in the network are well-defined in reality. Each confer- 
ence represents a community, and each independent team 
is counted as a separate community as well. 




■ NotinFBS 



(b)True Conference Alignment in 2003 



FIG. 5. (Color online) Maps showing the true conference 
alignment for FBS in 2009 (a) and the true conference align- 
ment for FBS in 2003 (b). The conferences are similar in these 
two years, but several teams move between conferences, and 
several more join the FBS. 

The conferences that existed in 2009 were the At- 
lantic Coast Conference (ACC), the Big 12, the Big 
East, the Big Ten, Conference USA (CUSA), the Mid- 
American Conference (MAC), the Mountain West Con- 
ference (MWC), the Pac-10, the Southeastern Confer- 
ence (SEC), the Sun Belt Conference, and the Western 
Athletic Conference (WAC). In 2009, there were three 
independent teams: Notre Dame, Army, and Navy. Sev- 
eral conferences that existed in 1995 (the first year of 
our analysis) are no longer in existence: the Big 8 (now 
the Big 12), the Big West, and the Southwest Confer- 
ence (SWC). Finally, there were 120 FBS teams in 2009, 
whereas there were 107 in 1995. The conference align- 
ment for 2009 and 2003 are shown in Fig. |5]as examples 
of the full landscape. All the same conferences existed in 
these two seasons, but a number of teams changed con- 
ferences, and four teams joined the FBS between 2003 
and 2009. 

The total number of teams in each conference for each 
season can be seen in Table [TTl Table [TTl reveals some 
basic information about the consistency of community 
structure between 1995 and 2009. For example, several 
conferences came into existence (CUSA, the MWC, and 
the Sun Belt), and several ceased to exist (the Big 8, the 
Big West, and the SWC). In addition, some conferences 
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changed in size (the ACC, the Big East, the MAC, and 
the WAC), while others remained constant (the Big Ten, 
the Pac 10, and the SEC). Finahy, the number of inde- 
pendents decreased fairly steadily over the time period 
in question. 

The mapping procedure for the simulated data was 
replicated here with q containing the latitude and lon- 
gitude of team i {i = 1,2,- • - ,120). Thus, the pixels were 
assigned to teams instead of cities. As before, the num- 
ber of pixels assigned to a single node varied greatly; 
community size in pixels does not necessarily indicate 
community size in number of teams. 

IV. RESULTS 
A. Simulated Data 

We conducted the analysis in Section II on the 30 sim- 
ulated networks to evaluate the effectiveness of the algo- 
rithm. For this analysis, we ran QCut 10 times for each 
realization {g — 10). The community sizes of the best 
results of QCut can be seen in Table |III[ included for 
comparison with the simulated community sizes as seen 
in Table [l| Here, we will look more closely at the scaled 
inclusivity of a representative network (network 30). The 
simulated communities can be seen in Fig. [4]^a), and the 
best QCut results for network 30 can be seen in Fig. |6] 
Note the misclassified nodes in the community contain- 
ing Texas in the QCut results (which are to be expected 
from any community structure algorithm). 




FIG. 6. (Color online) Network 30, with the nodes color-coded 
by community according to the best run of QCut. Note the 
misclassified nodes in Texas and Louisiana. 

The scaled inclusivity map using network 30 as the ref- 
erence partition was generated for both the simulated 
communities and for the results of QCut (Fig. [t]). Notice 
the similarities and differences between the two maps, es- 
pecially for the two nodes that were misclassified in net- 
work 30 (as seen in Fig. [6|. In the map made from the 
partitions generated by QCut, those nodes were not con- 
sistently classified in the communities they occupy in net- 
work 30, so their scaled inclusivity values were quite low. 
This would seem to indicate that these nodes were not 
at all consistently classified across networks. However, in 




(a) Scaled Inclusivity Across Simulated Communities 




(b) Scaled Inclusivity Across QCut Results 



FIG. 7. (Color online) Scaled inclusivity maps generated us- 
ing simulated communities (a) and the QCut results (b). For 
both maps, network 30 was used as the referent partition 
(maps in Fig. J4|. Note the effects of the misclassifications 
visible in Fig. iGron (b) here. 




(a) SI for Simulated Texas (b)SI for QCut Results on 
Community Simulated Texas Community 



FIG. 8. (Color online) Scaled inclusivity for the simulated 
community including Texas for simulated communities and 
the results of QCut. As before, both maps were generated 
with network 30 as the referent partition. Note the similarity 
of results in some regions and the vast differences for the two 
misclassified nodes from Fig. [6] 



the simulated communities, these nodes were fairly con- 
sistent in their classification. The community including 
Texas was isolated for both the simulated communities 
and the results of QCut (Fig. [s]) to make the differences 
more easily visible. Note that the nodes in central and 
south Texas have the highest scaled inclusivity values. 
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TABLE II. Conference sizes for all 15 years of the the college football network, shown to give the reader some idea about the 
overall community structure. 



Year 


ACC 


Big 8 


Big 12 


Big East 


Big Ten 


Big West 


CUSA 


MAC 


MWC 


Pac 10 


SEC 
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Sun Belt 


WAC 
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10 


2 
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8 
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12 


9 


10 


12 





8 


9 


4 
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12 


8 


11 





12 


12 


9 


10 
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8 


9 


4 


2007 


12 





12 


8 


11 





12 


13 


9 


10 


12 





8 


9 


4 


2008 
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8 
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12 


13 


9 
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9 
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2009 


12 





12 
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11 





12 


13 
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10 


12 
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3 



FIG. 10. (Color online) Weighted average of scaled inclusivity 
maps across all communities using QCut results. Note that 
the misclassified nodes do not significantly contribute to the 
final values for those nodes from any network when all possible 
community structures are considered. This indicates that the 
nodes are in consistent neighborhoods in all networks except 
the referent network (network 30) used in the analysis. 

region were classified together in the same community in 
all other networks, but due to imperfect overlap between 
their communities and the community shown here, their 
scaled inclusivity values are less than the optimal value of 
29. The other nodes shown here are sometimes included 
in that community and sometimes not, resulting in their 
lower scaled inclusivity values. 

In addition, the scaled inclusivity map containing both 
positive and negative values is shown in Fig. [9) This map 
appears very similar to Fig. [sj^b), but the negative values 
reveal more information. The node in central Texas has 
a very negative value - its absolute value is roughly equal 
to the nodes around it with positive values. The node 
in Louisiana, however, is not so negative. This indicates 
that the node in central Texas is consistently classified 




FIG. 9. (Color online) Scaled inclusivity including negative 
values for the simulated community including Texas using re- 
sults from QCut, generated using network 30 as the referent 
partition. Note that the central node that was misclassified 
is highly negative, indicating the likelihood that it is consis- 
tently in the same module as the nodes surrounding it for most 
other realizations in the group. The node in Louisiana that 
also has negative values has a much smaller absolute value, 
suggesting that it was not consistent in its classification across 
the other realizations. 



and the nodes around that region have lower values. The 
appendix contains images of all simulated communities 
that overlap with the Texas community for each realiza- 
tion to allow the reader to visually evaluate the consis- 
tency of the nodes in that community. The nodes in this 
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TABLE III. Community sizes and summed Jaccard index val- 
ues for the QCut results of all 30 simulated networks, listed in 
ascending order, to show the changes in community structure 
due to QCut. 
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with the other nodes around it in the other realizations 
of the network; the node in Louisiana, though, is incon- 
sistent in its classification across realizations. Thus, for 
the group, we can look at the node in Texas as belong- 
ing to the community containing the rest of central and 
south Texas across most subjects. The negative values 
reveal the distinction between the two misclassified nodes 
in this community. 

In a real data set, the true community structure is typ- 
ically unknown; the only data available are the results 
of a community detection algorithm, making misclassi- 
fied nodes such as those in Texas and Louisiana much 
more difficult to detect. Nodes that were misclassified or 
merely in a different community only in the referent par- 
tition appear to have very low scaled inclusivity values, 
as seen in Fig. p[b). However, in part (a), it can be seen 



that those nodes were more consistently classified in the 
simulated communites. In order to more effectively eval- 
uate the consistency of each node, irrespective of the spe- 
cific community assignment, we generated the weighted 



average map, which can be seen in Fig. 10 The Jaccard 
index value for each network, which was used to gener- 
ate the weights for the weighted average, is included in 
Table Hill Note that the nodes that were misclassified in 
network 30 have values that better match their neighbors 
and better refiect their consistency across all networks. 
This visualization may give better information about the 
consistency of individual nodes across multiple realiza- 
tions when the true community structure is unknown. 
There is one significant caveat to this approach. The 
weighted average uses a partition from every realization 
as a referent. As a result, there is no single underlying 
community structure, and its consistency cannot be eval- 
uated. However, the consistency of each individual node 
is better characterized. 




FIG. 11. (Color online) Map showing, for each node, the index 
of the realization that, when used as the reference partition, 
yielded the highest scaled inclusivity value. In other words, 
this indicates which subject's scaled inclusivity map is most 
representative for each node across the entire group. 



Fig. 



11 shows another map of the USA. In this case, 
every partition was used as the referent partition Qi?, 
and a map was made showing which partition yielded 
the highest scaled inclusivity value for each node. This 
map reveals which referent partition best characterizes 
each node or region. This map is useful when interpreting 
the negative scaled inclusivity maps together with the 
weighed average map. Regions of consistent community 
structure can be identified on the weighted average map. 
To then evaluate the organization of a community in any 
particular region, one would consult a negative scaled 
inclusivity map. However, as negative scaled inclusivity 
maps, by necessity can only be generated for one commu- 
nity of a particular referent partition, the "best" referent 
needs to be identified. Maps showing the referent that 
had the highest scaled inclusivity value (Fig. 11 ) are then 
consulted to pick the most appropriate partition to ex- 
amine. For example, the northeastern United States has 
relatively high values, and the region is best character- 
ized by partition 1. The negative scaled inclusivity map 



for that region can be seen in Fig. 12 
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FIG. 12. (Color online) Scaled inclusivity including negative 
values for the simulated community including the northeast- 
ern United States, using partition 1 as the referent partition 
Qr. Most of the green (light gray) values in this image are 
very close to 0; some nodes farther west had very small values, 
but they were not pictured for space reasons. 



B. Real Data 

We again conducted the analysis as described in Sec- 
tion II (with g = 10) to get the best partition from 
QCut (Q-) for every year under consideration for the 
college football networks. The community structure of 

It 



the best run of QCut for 2009 is visible in Fig. 13 



should be noted that this partition is not the same as the 
true conference alignment. The football conferences form 
well-defined communities in general, but a network based 
solely on games played may result in partitions that do 
not directly match the conferences. For example, inde- 
pendent schools are put into a larger community, in part 
because they tend to play a large number of schools from 
one conference. It some cases, teams from two geograph- 
ically close conferences played a large number of inter- 
conference games. An optimal partition of this network 
lumps two such conferences together into one community. 
This occurred during multiple seasons with the Sun Belt 
and the SEC as well as with the MWC and the WAC. 

The overall scaled inclusivity map was generated for 
the true conference alignment and the results of QCut, 



and the results are visible in Fig. 14 The consistency is 
significantly higher in the network showing the true con- 
ference alignment. This is due to differences between the 
true conference membership and the community struc- 
ture identified using QCut. 

For time series data, there is likely a single network 
(probably the first or last realization, although not nec- 
essarily) that is of greatest interest. In networks with 
linear order, the scaled inclusivity for the most recent 




FIG. 13. (Color online) The best partition from the QCut 
runs for 2009. 




(a) Scaled Inclusivity for True Conference Alignment 



f 




(b) Scaled Inclusivity for QCut Conference Alignment 



FIG. 14. (Color online) Scaled inclusivity map for FBS con- 
ferences, with 2009 as the referent network for comparison. 
Shown for true conference alignment in (a) and the QCut 
approximations in (b). 



realization shows how consistently the community struc- 
ture over time aligns with the current or most recent 
version of the network. However, the overall character- 
ization of the group of networks may be of interest as 
well. The scaled inclusivity of the FBS networks was 
shown for 2009 because it is the most recent network; 
we have also included the weighted average map for FBS 
(Fig. 15) and the maximum scaled inclusivity map (Fig. 
16). Interestingly, the weighted average map is very sim- 
ilar to the map for 2009. Of even more interest is that 
the maximum scaled inclusivity values appeared mostly 
in the first few networks. It should be noted here that 
because of how this was calculated, in the event of equal 
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FIG. 15. (Color online) Weighted average map for the scaled 
inclusivity of the FBS conferences, shown for the results of 
QCut. 




FIG. 16. (Color online) Year with maximum scaled inclusivity 
for each node. Network 1 here represents 1995, and network 
15 represents 2009. 

maximum scaled inclusivity values, the earliest network 
in the time series will be chosen. This may be the case 
here for conferences that remained unchanged through- 
out the time period studied, such as the PAC 10 and the 
Big Ten. 

V. DISCUSSION 

This paper has introduced scaled inclusivity, which is 
a new method for evaluating the consistency of the par- 
titioning of related networks. There are several methods 
for determining the community structure of a network; 
the best way to partition a network is an area of ongo- 
ing research. The comparison of community analyses has 
recently become of interest as well. Emphasis has been 
placed on the change in community structure over time 
as various datasets evolve. 

Several methods have been proposed for the compari- 
son of community structure in different networks. Palla, 
et. al. [6 studied the relationships between size, age, 
stationarity, and lifetime of communities. They investi- 
gate the characteristics of communities with short and 
long lifespans in changing networks. The stationarity 
is defined as the average consistency from each time 
step to the next. They propose a method for determin- 



ing the best match among communities in consecutive 
time steps; we consider every pair of communities with 
any overlap when calculating scaled inclusivity. Asur et. 
al. [8] define community events (continue, merge, split, 
form, and dissolve) and node events (appear, disappear, 
join, and leave). They captured the occurrence of these 
events over time and used that information to charac- 
terize the dynamic community structure of the network. 
Chakrabarti, et. al. [7j define a method for evolution- 
ary clustering, which seeks to balance the quality of fit 
for any specific network with similarity to previous and 
subsequent partitions. Fenn, et. al. [9j track community 
changes by using centrality metrics to analyze the roles 
of individual vertices in economic networks. As in scaled 
inclusivity, two communities at different times are com- 
pared if they have any common nodes; the method does 
not attempt to select the best matching communities. 
These methods are designed specifically to characterize 
change over time but are not well suited to cross-sectional 
analysis. 

Hopcroft, et. al. [5 take a different approach. They 
use a community detection algorithm that finds the "nat- 
ural communities" of a network. This analysis leaves 
some nodes out as not belonging to any community. 
Then, to compare across time, they use a metric they 
call the best match, which is defined as follows for two 
communities C and C: 

match{C, C) = mm f ' ' , ^^.^ ' 1 • (4) 

In that paper, they investigated citation networks from 
the NEC CiteSeer database at two different time points. 
Because citation networks never lose edges and because 
only two realizations were investigated, they manually 
compared the best match communities from the two 
datasets, noting the growth and emergence of commu- 
nities over time and the corresponding changes in the 
fields they characterize. This method could be applied 
on longitudinal or cross-sectional datasets, provided that 
the number of networks is limited. 

To our knowledge, this is the first methodology to allow 
a network- wide unbiased comparison of community struc- 
ture across multiple realizations of a network. Scaled in- 
clusivity differs in several ways from other approaches 
for evaluating the consistency of community structure. 
In scaled inclusivity, the partitions of multiple networks 
are compared to a reference partition with no a priori as- 
sumptions about which communities match one another. 
Every node is independently assigned a scaled value for 
each comparison to the reference partition based on how 
well the communities containing that node match in the 
two partitions. It is thus possible to quantify the con- 
sistency of a node's classification across the partitions of 
several networks. One advantage of this approach is the 
inclusion of all nodes in the final analysis. Instead of 
only indicating the most consistent communities, scaled 
inclusivity assigns a consistency value to every node in 
the network. In addition, because the method makes no 
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assumptions about the relationship among the networks, 
scaled inclusivity is equally applicable to cross-sectional 
(across-network) and longitudinal (across time) analyses. 
Finally, scaled inclusivity can be applied to datasets of 
any size. 

We tested our algorithm on one simulated dataset and 
one real dataset. The simulated data used randomly 
generated communities which were manually assigned to 
the most populous cities in the United States to yield 
geographically contiguous communities. We found that 
the areas with the highest scaled inclusivity values (and 
thus the highest consistency of classification) were Texas, 
Florida, the Northeast, Southern California, and North- 
ern California. Florida and the Northeast are highly con- 
sistent because of the geographic constraints placed on 
the community definitions when generating the networks. 
The most natural way to put these regions into geograph- 
ical communities is to start from the edges of the map and 
work inward. For both California regions, the density of 
nodes determined the consistency of classification. The 
generally limited community size and the concentration 
of cities in Southern California caused those cities to be 
placed in the same community in nearly every map. A 
similar effect can be seen in Northern California as well. 
While there is not such a simple explanation for the high 
scaled inclusivity in Texas, the central region of Texas 
that has the highest consistency is grouped together in 
every partition, as can be seen in the Appendix. In ad- 
dition, the Midwest, Southeast, and West varied more in 
their community boundaries due to changing community 
size; this is refiected in their lower scaled inclusivity val- 
ues. Overall, the data seem to refiect the consistency of 
the partitions. 

We also compared college football networks for the 
years 1995 to 2009. The scaled inclusivity for the true 
conference alignment closely follows what would be ex- 
pected. The SEC, PAC-10, and Big Ten teams ah had 
the highest possible scaled inclusivity values, which is 
to be expected because these conferences did not change 
membership during the time period studied. The teams 
in the Big 12 had only slightly lower values; the additions 
to the Big 8 in 1996 made a small but noticeable impact 
on the consistency of the conference as a whole. Confer- 
ences that have changed drastically since 1995 have much 
lower values, including the Big East, the WAC, Confer- 
ence USA, and the Sun Belt. As with the simulated data, 
the scaled inclusivity values for the college football net- 
work fit with the known changes in community structure. 

The results were less promising for the analysis of the 
football network performed on the QCut results. Due 
to the misclassifications of some teams (especially inde- 
pendents) and the combinations of conferences in certain 
years, the most consistent conferences are not so easily 
identified. Some of these problems are probably not a 
result of the community structure algorithm but rather 
of the network itself. Independent teams frequently play 
many teams from conferences located nearby, and confer- 
ences with overlapping territory also form rivalries with 



many inter-conference games. As a result, the inclusion 
of independent teams and the combination of multiple 
conferences is to be expected. The scaled inclusivity val- 
ues are still generally lower for the less consistent confer- 
ences, but the differences are less clear. The results of 
this analysis are more difficult to interpret than similar 
results from the true conference alignments. 

Among the difficulties inherent in identifying consis- 
tency or change in network community structure is the 
limitation of current algorithms. Because finding a net- 
work partition with optimal modularity has been shown 
to be an NP-complete problem, any algorithm based on 
modularity that runs in a reasonable amount of time will 
yield imperfect results. In many cases, the solution given 
may vary across multiple runs on the same network. In 
addition, there is evidence to suggest that the problem 
of optimizing Q has a degenerate search space, meaning 
many solutions exist with near-optimal values of Q that 
have significantly different community structure [15 . To 
mitigate these effects on our analysis, we ran the modu- 
larity algorithm (QCut) multiple times for each network 
and selected the best results. This prevents a single im- 
probable run from skewing the results of the analysis. 
However, we have seen that imperfections in the reference 
partition can unduly inffuence the final scaled inclusivity 
values in situations where a node is consistently classified 
in all other networks. To counter this problem, we gen- 
erated a weighted average of the scaled inclusivity with 
each partition as the referent partition, a map that uses 
negative values to evaluate the consistency of nodes not 
included in the referent's community, and a map showing 
which referent partition yielded the highest scaled inclu- 
sivity value for each node independently. All of these 
methods minimize the bias from a single partition on the 
final evaluation of consistency. This was demonstrated 
in greater detail with the community containing Texas 
in Section IV. 

In this paper, we propose a method for comparing the 
consistency of community structure across different real- 
izations of a network, be they the same network over time 
or multiple realizations of a single network. In particu- 
lar, we propose a method to describe how consistently 
each node is part of the same community across different 
partitions using a metric called scaled inclusivity. This 
method enables us to identify which nodes tend to remain 
in the same community in different partitions, forming a 
"core" of that community. Likewise, the method also al- 
lows identification of nodes that become part of different 
communities across partitions. 

Community structure analysis is a growing field, and 
new algorithms to find the optimal partition of a network 
are being developed. Improved algorithms promise bet- 
ter partitioning of complex networks, which would ren- 
der our method more effective in analyzing the consis- 
tency of the true community structure across networks. 
In addition, some recent community structure algorithms 
permit a node to be a member of multiple communities 
[ini [17] . This is thought to better characterize the com- 
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munity structure of some real-world networks. Basing 
our analysis on such algorithms is an avenue for poten- 
tial future research. Finally, a key future application 
of the scaled inclusivity algorithm is its use on biologi- 
cal networks. The study of networks across subjects in 
research studies is rapidly growing, and the ability to 
compare the community structures of different popula- 
tions promises to be a valuable research tool. Some of 
the growing subfields of biological network science in- 
clude complex brain networks, protein networks, genome 
networks, and metabolic networks. 
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FIG. 17. (Color online) This figure shows the simulated communities for each network that overlap with the community 
containing Texas in the reference community (network 26). That community is outlined in each image. The colors of the other 
communities in each image are arbitrarily assigned. 
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